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ABSTRACT 



In this readership study of the Review of Educational 
Research , approval and use of the 15 issues within the 
three-year cycle from June, 1965 to June, 1968 are 
determined for each issue separately and for all the 
issues as a set. The survey respondents, who are all 
members of the American Educational Research Association, 
teach, administer, consult, and engage in research projects 
Half have PhD’s and most are authors of papers and 
journal articles as well as active readers of other 
journals. They advocate more frequent review of several 
topics, which happen to be topics of the issues that are 
used the most. Respondents propose changes to improve 
the writing style, editorial quality, and scope of the 
Review. Half would prefer an annual review of educational 
research to the current Review. Overall use of the 
journal is moderate, partly because each educational 
researcher tends to read issues only in his areas of 
specialization. The Review is used for several different 
purposes; two major uses are for current awareness and 
for learning about areas peripheral to one's specialty. 

In addition, the Review acts as a stimulus to informa- 
tion-seeking in connection with the publications cited. 

The author recommends changes in format and in content 
so that each issue would appeal to specialists on the 
topic reviewed and also to a more general audience 
attracted by current awareness features designed 
especially for them. 
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INTRODUCTION 



An aid to professional communication such as the Review of 
Educational Research can be studied from many different points of view 
and with many different goals. The various points of view include 
impact and effectiveness, and the goals include making editorial or 
technical changes. This study of the Review seeks to understand both 
the impact and the effectiveness of the journal, with the goal of recom- 
mending policies that will enhance it as a communication tool. 

Measuring the effectiveness of a communication technique means 
measuring how well it does the job for which it was intended. Measuring 
the impact means measuring changes in communication patterns or habits 
that may accompany use of that particular tool. For example, the effec- 
tiveness of the Review was measured in this questionnaire survey in two 
basic questions: 

•How much did respondents use each issue? 

•What did they use the issues for? 

Two-way contingency tables were used to analyze further the journal’s 
effectiveness — e.g., to see how useful for counselors the issue on 
guidance and counseling was. The impact of the Review was measured by 
asking respondents what happened as they read each issue — that is, what 
impact each issue had on their behavior: 

•Did they recognize cited publications from 
previous reading? 

•Did they try to contact an author of a cited 
publication? 

These and the other questions asked represent only one approach to 
measures of the effectiveness and impact of the Review. A policy change 
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that would improve the journal should be based on information from many 
sources, but these measures of actual use and impact can lend a strong 
base of information to the decision— making process • This study answers 
two broad policy questions: 

•How can the Review be improved? 

•How desirable is an annual review of educa- 
tional research? 

The validity of the survey itself rests on recognizing the impor- 
tance of professional journal literature and reviews. Joseph Kuney, in 
the 1968 Annual Review of Information Science and Technology . 1 properly 
labels the printed journal as "the most widely used tool offered by the 
present system of information transfer." Kuney implies that the planning 
and producing of journals to serve various purposes and to satisfy user 
needs is an art. According to a related study by Gannett, the frighten- 
ing rate of literature growth in all fields will result in: 

an increasing emphasis on secondary publication services, 
such as abstract journals, indexes, current awareness 
bulletins, and on any other information services that 
will enable the individual to feel that he is somehow 
coming to grips with the literature. 

Even though Kuney agrees with the urgent need for these secondary 

services and the need for critical reviews and progress reports, he is 

also concerned with maintenance of journals that serve multiple functions. 

For example, this study found that the Review of Educational Research 

serves current awareness and reference functions, among others. Most 

^uney, Joseph H., "Publication and Distribution of Information," 
in: Cuadra, Carlos A,, ed., Annual Review of Information Science and 

Technology , Vol. Ill, (Chicago: Encyclopaedia Britannica, Inc.), 1968. 

2 

Gannett, K.K., "Technical Journals and the Information Explosion," 
in Proceedings of the 14th International Technical Communications 
Conference (Washington, D.C.) 1967. 
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other information tools do not fill multiple roles. Kuney concludes 
that higher selectivity in journal contents might be best effected at 
a "mission-oriented" level, while the role of the discipline-oriented 
publisher would be to "wholesale" papers to mission-oriented informa- 
tion centers. The importance of critical reviews for condensing and 
organizing the huge volume of education research information and also 
for serving more than one function cannot be denied. 

What kind of an educational research review is optimal? The 
advantages of the Review of Educational Research , which covers one 
topic area on the average of every three years, must be compared with 
the advantages of an annual review of educational research. The 
decision to publish an annual review entails considerations of the 
growth rate of literature in the various topic areas to be reviewed 
and the popularity and relevance of topics for different user groups. 
Several questions in this survey investigated these factors- 

Apart from the quandary of coping with the infamous "information 
explosion," there are problems specific to information exchange in the 
field of education, whose information users include everyone from U.S. 
Office of Education administrators and Title III directors to university 
professors and (a few!) elementary school teachers. As a result, 
journals in the field of education are diverse in scope and sophisti- 
cation. The Review of Educational Research , with its cycle of 15 topics, 
is tackling the scope problem at a single level of sophistication. 

Changes for the Review suggested in the survey encompass both 
scope (changes in the frequency of review of certain topics and the 
addition and deletion of others) and sophistication (suggestions for 




’’more critical" reviews instead of reviews that seem to be merely 
bibliographies of the current literature) . 

With this setting for the survey and statement of the problems, 
let us inspect the procedure and results. 
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I. DESCRIPTION OF THE QUESTIONNAIRE 

The two policy questions mentioned above — how to improve the Review, 
and how desirable is an annual review of educational research — encompass 
a host of smaller questions that also were included in this survey. 

For example, knowledge of how much the Review is used, what it is used 
for, and what types of readers use it is necessary to answer both the 
policy questions. 

A four-page questionnaire (which is appended to this report) was 
sent to 1006 non-ins t it utional members of the American Educational 
Research Association (AERA) ; they represented a systematic sample of 
one-eighth of the total AERA membership. After two mailings, the 
response rate was 60%, or 607 questionnaires, of which 572 had enough 
questions answered to be useable. These 572 compose the data set used 
in this analysis. 

The questions require respondents to think back over the past three 
years and remember whether and how they used specific issues of the 
Review. This is a difficult task; lapse of memory may be responsible 
for the high rate of non-response to some of the questions. 

Each of four basic questions had to be answered for each of the 15 
issues of the Review from June, 1965 to June, 1968, a three-year cycle. 
We shall label these four questions: frequency-of -review, extent-of- 

use, purposes-of-use , and results-of-use. 
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II. CHARACTERISTICS OF QUESTIONNAIRE RESPONDENTS 

Knowledge of the demographic traits of respondents is necessary 
to fully analyze their use of the Review and their non-use and to 
posit explanations for that behavior. Appendix I shows results 
of the demographic questions asked on the back page of the question- 
naire. 

The survey respondents represent an elite group of educators 
in teaching, research, and administrative settings. Research or 
administration is the primary job activity for 41%, and teaching is 
the primary activity for 39%. Not unexpectedly, 68% are employed in a 
college or university. Research methodology is the primary professional 
specialty for 23% of the respondents, and another 12% are primarily 
engaged in educational administration. Furthermore, almost half of the 
respondents received their highest degree from an institution classified 
by the 1966 Carrter report"^ as a top quality institution in graduate 
education. Finally, the highest earned degree is a PhD (or MD) for 
44% of the respondents. Almost 60% of these highest degrees were earned 
after 1961; the fields represented by the largest respondent groups are 
education (32%) , educational psychology (15%) , and educational administra- 
tion (15%). 

The respondents are unusually active journal readers and authors of 
papers and articles. A quarter of the respondents claim to read seven 



^The Carrter Report : An Assessment of Quality in Graduate Education , by 

Alan M. Carrter, (Washington, D.C.: American Council on Education, 1966). 



0 







or more journals regularly, and an additional 47% read four to six 
regularly. They are also prolific authors: 42% have written five 

or more unpublished papers or reports within the past five years, and 
26% have published four or more journal articles within that time 
span. Thirteen percent have also written at least one chapter in a 
book, and 13% also have written at least one book over the past five 



years . 
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III. USE OF THE REVIEW 



Findings from Four Basic Questions on: 

A. Ideal Frequency of Review 

B. Extent of Use 

C. Purposes of Use 



D. Consequences of Use: Impact of 

the Review 






A. FREQUENCY OF REVIEW 



Basic Results 



This is the text of the frequency-of-review question: 



HOW OFTEN SHOULD THIS TOPIC BE REVIEWED? 



(Circle the most appropriate choice.) 



This topic should be: 



1 — retained as is (reviewed every 3 years) 

2 — reviewed more often 

3 — reviewed less often 

4 — excluded from the Review 



5 — modified or subdivided: Please list new 

titles in question 4 below 



(The titles of the issues were listed in reverse chronological order, 
with the numbers 1 through 5 opposite each title.) The new titles 
that were suggested are summarized in Section V: How to Improve 



the Review. 



Table I shows the replies to this question for the 572 respondents 
The popularity of the issues, or how often they should be reviewed, 
varies extensively. For example: 54% of the respondents would like 

to see "Instructional Materials" reviewed more often than every three 



years, but only 12% would like to see "International Development 
Education" reviewed more often. The percentages in the Reviewed Less 
Often column are complementary to these: only 3% would like to see 
"Instructional Materials" reviewed less often, while 23% (the highest 
percentage in that column) think that "International Development 
Education" should be reviewed less often. About half the replies for 
each issue favor a Retained As Is policy, and very few favor the 



Excluded or Modified options 
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We can discuss these results in Table I in two ways: by talking 

about the average response percentage for each choice in the frequency- 
of-review question, and thus the average or typical issue; and by rank- 
ing the titles according to their relative popularity — that is , those 
that should be reviewed more and less often. The ranks of the titles 
are shown in Table II. The average response percentages for the 
typical issue are presented below. 




Average Frequency: The Typical Issue 

These are the average percentages of response (over 15 issues) 
for each portion of the question: 



HOW OFTEN SHOULD THIS TOPIC BE REVIEWED ? THIS TOPIC SHOULD BE: 



retained as is (i.e., reviewed every 3 years) 

reviewed more often 

reviewed less often 

excluded from the Review 

modified or subdivided 



47% 

30 

11 

2 

1 



average non-response 



100 % 



As might be expected, almost half the survey respondents, on the 



average, think that the typical issue should be retained in a three- 
year cycle. Motivation to retain the status quo, whether due to lack 
of imagination or lack of energy, is always strong. However, the 
fact that an average of 30% think that the typical issue should be 
reviewed more often and only 11% think that it should be reviewed less 
often is noteworthy. The former, and larger, group of educational 
researchers feels that developments in many areas are rapid enough to 
warrant more frequent attention and review. Finally, the low response 
percentages for the Excluded and Modified choices, viewed along with 
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the high percentages for the other three choices, may be interpreted 



as the reflection of general interest in the topics as they exist 



and as a preference to alter the cycle rather than the contents of 
individual issues. 



Relative Frequency; Ranking the Issues 

Table II shows the ranking of titles according to their popularity, 



or how often they should be reviewed. It is interesting to note that 



the most popular issue and the least popular issue are the two most 
recent ones published (at the time the questionnaire was distributed) . 
"Instructional Materials" was published four months prior to distri- 
bution of the questionnaire, and "International Development Education" 
two months prior. (The impact of "International Development Education" 



may not have been felt yet, since this was the inaugural issue.) 

The reasons for preferences regarding frequency of review undoubtedly 
reflect respondents’ perceptions of the rate of literature growth in 



each topic area, their individual interests in that area, and the 



availability of reviews and information about that topic in other 
professional journals and published materials. For example, "Education 
for Socially Disadvantaged Children," one of the most popular 
issues, currently is an active research area, and many educational 
researchers could easily profit from an overview of the topic. On 
the other hand, "Educational Organization, Administration, and 
Finance" is also a crucial area in the general field of education 
todays but only the sub-group of respondents who specialize in that 
area seem to use this issue of the Review (as will be shown later) . 
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Frequency: An Overview 

Instead of ana.1.yzing the popularity of each individual issue, 
let us consider respondents’ satisfaction with the entire set of 
15 issues that have appeared over the past three years. Table III 
shows the cumulative response percentages for the number of issues 
that should be retained, reviewed more often, and so forth. For 
the Retained portion of the question, a third of the respondents 
think that no more than six issues should be retained in a three- 
year cycle. Half think that no more than eight (or about half the 
issues) should be retained. In the same manner, about three-fourths 
think that no more than 10 issues should be retained. In other 
words, the bulk of these respondents would like to see the cycle 
of at least five issues changed, and half the respondents would 
like to see the cycle of at least seven issues changed. This is a 
considerable mandate to the editor! 

The type of change desired by most respondents is a change in 
the frequency of review and not exclusion or modification of the 
topic, as can be seen in Table I. The specific titles of the issues 
that should be reviewed more or less often are found in Table II. 
Table III, though, gives an overview. 

From this table, it is evident that, considering those who 
replied to the Reviewed More Often portion of the question, as many 
as a quarter favor reviewing at least seven topics more often— this 
is about half the number of issues in a three-year cycle. In 
addition, almost half the respondents think that at least five topics 
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should be reviewed more often. Most think that at least one or two 



topics should be reviewed more often, but this is not too startling 



because these topics could easily be the one or two areas of 



specialization for each respondent. 

Respondents aren’t as inclined to request less frequent 



appearances of some issues. In fact, only about half of all the 
572 respondents want to see even one issue less often. This probably 



means that a researcher sees no logical reason for the amount of 
information available to him to be reduced. Only 16% of the Less 



Often group think that any more than five issues should be reviewed 



less often — a small percentage indeed. 

Finally, only 94 people give any response at all to the Excluded 

choice, and 70 to the Modified category. And the number of issues 

that most of these respondents think should be excluded or modified 

is only one or two. These results might be explained by the fact 

1 

that excluding a topic is a rather severe fate that respondents might 
be reluctant to recommend, and modifying a topic might require too 



much respondent effort in creating a new title. 
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